A datacenter may include a plurality of computers (e.g., a computing cluster) that are connected to a network. The plurality of computers may be located behind a firewall. In general, a computer of the cluster may initiate a request for an object from a server via the network, but the server cannot initiate a request for an object from one of the computers in the cluster because the firewall protects the computing cluster from unauthorized incoming requests.
One conventional solution is to use websockets to enable bidirectional communications between the server and one of the computers of the computing cluster. Websockets provide a persistent connection between a client and server that both parties can use to start sending data at any time. The server may receive a plurality of asynchronous requests. The server must maintain a mapping to respond to the plurality of asynchronous requests such that the replies are sent back to the correct requesting entity. The server may maintain a plurality of websockets. The server may receive a large number of requests (e.g., a million) via the plurality of websockets. The server must map and maintain a large database to respond to the large number of requests. A large amount of memory and computing resources may be required to maintain the large database.
Another conventional solution is to open multiple connections between the server and the client. This may allow the server to manage multiple requests. However, many of the connections may be underutilized because the server has supplied more connections than in demand. The number of open connections may be modified, but such a modification may require a monitor to analyze historical connection data and to predict the number of connections needed at a particular instance.